Content Presentation and Interaction Across Multiple Displays

ABSTRACT

A first electronic device with a touch-enabled display establishes a wireless connection with a second electronic device that controls display of a user interface on a second display. The first electronic device displays a first user interface on the touch-enabled display, the first user interface including a first affordance corresponding to first media content. While displaying the first user interface, the first electronic device detects a first user input at a location on the touch-sensitive display that corresponds to the first affordance in the first user interface. In response to detecting the first user input, the first electronic device transmits, to the second electronic device via the wireless connection, instructions enabling display of at least a portion of the first media content on substantially the entire second display controlled by the second electronic device.

RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.16/378,436, filed Apr. 8, 2019, which is a continuation of U.S.application Ser. No. 15/174,808, filed Jun. 6, 2016, now U.S. Pat. No.10,254,924, which is a continuation of U.S. application Ser. No.13/598,574, filed Aug. 29, 2012, now U.S. Pat. No. 9,360,997, all ofwhich are incorporated herein by reference in their entireties.

BACKGROUND

The present disclosure relates in general to presenting information inan electronic environment and in particular to content presentation andmanipulation using multiple electronic devices.

As screen real estate becomes increasingly valuable, systems have beendeveloped to use a secondary display screen to either mirror, or provideadditional display area to, a primary display screen of a computingdevice. In the mirroring scenario, what is displayed on a primarydisplay screen of a computing device (e.g., a laptop) may be “mirrored”by a secondary display screen (e.g., attached monitor) that is, forinstance, larger and capable of higher-resolution display. In theextension scenario, the secondary display simply provides additionalscreen real estate but does not “mirror” the content of the primarydisplay. For examples, a user may use the primary display to view onewebsite and use the secondary display to view another website.

However, conventional approaches for using a primary display with asecondary display have shortcomings. First, users have to manuallyconfigure the settings of the primary and secondary displays, a processthat can confuse and intimidate a significant number of users. Second,even for users who are capable of performing such configuration, theprocess is time-consuming, inefficient, and irritating to the users'overall computing experience. Third, even technically savvy users mayfail to discover display arrangements and/or settings that wouldmaximize the advantage of having two displays.

SUMMARY

Certain embodiments of the present invention relate to contentpresentation across multiple electronic devices. In some embodiments,content of a software application can be presented across the displaysof multiple electronic devices in an intuitive and coordinated fashion.User interfaces displayed by the displays of the electronic devices mayrepresent different aspects or different levels of a set of informationitems in the software application. In response to receiving a user inputvia a user interface of an electronic device, another user interface ofanother electronic device may be modified based on the user input. Forexample, a tablet computer may display a user interface including a listof songs for a media player application and a desktop monitor maydisplay another, distinct user interface including detailed informationfor a particular song from the list, in response to a user selection ofthe particular song via the user interface.

Certain embodiments of the present invention enable coordinatedinteraction and presentation across a touch device and a non-touchdevice. A touch device (also referred to as touchscreen device) can beany electronic device that can receive user input via a touchscreen or adisplay of the device, such as a tablet computer or a smartphone. Anon-touch device (also referred to as display device) can be anyelectronic device that cannot receive user input directly through adisplay coupled to the device, such as a conventional light emittingdiode (LED) computer monitor or a liquid crystal display (LCD)television screen. In some embodiments, a touch device can establish aconnection with a non-touch device. At least one of the devices canexecute an application where content of the application may be presentedacross the displays of the devices. A set of user-selectable userinterface items can be displayed on the touch-enabled display of thetouch device. The electronic device can receive a user selection of oneof the set of user interface items displayed on the touch-enableddisplay. In response to receiving the user selection, another set ofinformation items can be presented on the non-touch device. In someembodiments, the other set of information items represents an additionallevel of information related to the selected user interface item.

The following detailed description together with the accompanyingdrawings will provide a better understanding of the nature andadvantages of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example of a system according to some embodimentsof the present invention.

FIG. 2 illustrates a computer system according to an embodiment of thepresent invention.

FIG. 3 illustrates a simplified block diagram of a system similar toFIG. 1 in accordance with some embodiments of the present invention.

FIG. 4 illustrates another simplified block diagram of a system similarto FIG. 1 in accordance with some embodiments of the present invention.

FIG. 5 illustrates an example of a process performed by a touch devicefor interacting with a non-touch device according to some embodiments ofthe present invention.

FIG. 6 illustrates an example of a process performed by non-touch devicefor presenting information across the non-touch device and a touchdevice in accordance with some embodiments of the invention.

FIGS. 7-17B illustrate examples of user interfaces generated for displayacross a touch-enabled display of a portable electronic device and anon-touch display of an electronic device.

DETAILED DESCRIPTION

In the following description, for the purposes of explanation, numerousdetails are set forth in order to provide an understanding of variousembodiments of the present invention. It will be apparent, however, toone skilled in the art that certain embodiments can be practiced withoutsome of these details.

Certain embodiments of the present invention relate to contentpresentation across multiple electronic devices. In some embodiments,content of a software application can be presented across the displaysof multiple electronic devices in an intuitive and coordinated fashion.User interfaces displayed by the displays of the electronic devices mayrepresent different aspects or different levels of a set of informationitems in the software application. In response to receiving a user inputvia a user interface of an electronic device, another user interface ofanother electronic device may be modified based on the user input. Forexample, a tablet computer may display a user interface including a listof songs for a media player application and a desktop monitor maydisplay another, distinct user interface including detailed informationfor a particular song from the list, in response to a user selection ofthe particular song via the user interface.

Certain embodiments of the present invention enable coordinatedinteraction and presentation across a touch device and a non-touchdevice. A touch device (also referred to as touchscreen device) can beany electronic device that can receive user input via a touchscreen or adisplay of the device, such as a tablet computer or a smartphone. Anon-touch device (also referred to as display device) can be anyelectronic device that cannot receive user input directly through adisplay coupled to the device, such as a conventional light emittingdiode (LED) computer monitor or a liquid crystal display (LCD)television screen. In some embodiments, a touch device can establish aconnection with a non-touch device. At least one of the devices canexecute an application where content of the application may be presentedacross the displays of the devices. A set of user-selectable userinterface items can be displayed on the touch-enabled display of thetouch device. The electronic device can receive a user selection of oneof the set of user interface items displayed on the touch-enableddisplay. In response to receiving the user selection, another set ofinformation items can be presented on the non-touch device. In someembodiments, the other set of information items represents an additionallevel of information related to the selected user interface item.

Generating and presenting different levels of information acrossmultiple electronic devices and in a display arrangement intuitive to auser can enhance the user's experience. For example, the overall userexperience can be optimized by coordinating and presenting informationitems associated with an application across the different devices in away that maximizes use of a display device with a large display area anda smaller touchscreen device.

FIG. 1 illustrates an example of a system environment 100 according tosome embodiments of the present invention. System environment 100includes a touchscreen device 102 communicatively coupled to a displaydevice 104. Touchscreen device 102 can be an electronic device that iscapable of sending and receiving content, and displaying a visualrepresentation of the content on a display of the device. For example,touchscreen device 102 can be a tablet computer, a smartphone, a mobilephone, etc. In some embodiments, touchscreen device 102 can have a userinterface that includes user input and/or output devices such as atouchscreen 106. Touchscreen device 102 can also include any number ofother user input and/or output devices such as buttons, virtual orphysical keyboards, microphones, speakers, and the like. In someembodiments, touchscreen device 102 can receive user input through itsdisplay via hand gestures (i.e., with or without the user physicallytouching the display).

Display device 104 can be an electronic device capable of sending,receiving, and displaying content. Display device 104 can be a desktopcomputer, a laptop computer, a monitor, a tablet computer, a mobilephone, etc. In some embodiments, display device 104 can include anon-touch display. In other embodiments, display device 104 can have atouch-enabled display. Display device 104 can also include a number ofinput and/or output devices such as keyboards, speakers, microphones,etc.

As shown in this example, touchscreen device 102 and display device 104can simultaneously display multiple, distinct user interfaces for asingle software application on each of their display devices. Anapplication executing on one of the two devices can generate a userinterface for display on the local device and generate another userinterface for display on the other device. In one embodiment, the userinterfaces generated can be mirror images of each other orpartial-mirror images. In some embodiments, one of the user interfacesmay include a set of information items and the other user interface mayinclude another set of information items that includes a differentabstraction level compared to the first set. Some embodiments allow oneof the user interfaces to include a set of controls that can control ormodify content displayed on the other user interface.

In response to receiving user input via the user interface ontouchscreen device 102, the application in some embodiments can modifythe content of the user interface being displayed on display device 104.For example, an e-mail application executing on touchscreen device 102can cause both touchscreen device 102 and display device 104 to displaya set of e-mails in an inbox. In response to receiving a user selectionof a particular e-mail from the set of e-mails displayed on thetouch-enabled display of touchscreen device 102, the e-mail applicationmay cause display device 104 to display the contents of the particulare-mail while a view of the list of e-mails is maintained on touchscreendevice 102. As such, the user can select any e-mail from the list ofe-mails displayed on touchscreen device 102 while simultaneously viewingthe contents of the selected e-mail on display device 104. Theapplication can generate different user interfaces to be displayedacross devices 102 and 104 in a coordinated fashion and in response touser input received via the user interface on touchscreen device 102.

FIG. 2 illustrates a computer system 200 according to an embodiment ofthe present invention. Computer system 200 can be implemented as any ofvarious computing devices, including, e.g., a desktop or laptopcomputer, tablet computer, smart phone, personal data assistant (PDA),or any other type of computing device, not limited to any particularform factor. Portable electronic device 102 and electronic device 104 ofFIG. 1 can include similar components as those shown in computer system200. Computer system 200 can include processing unit(s) 205, storagesubsystem 210, input devices 215, display 220, network interface 225,and bus 230.

Processing unit(s) 205 can include a single processor, which can haveone or more cores, or multiple processors. In some embodiments,processing unit(s) 205 can include a general-purpose primary processoras well as one or more special-purpose co-processors such as graphicsprocessors, digital signal processors, or the like. In some embodiments,some or all processing units 205 can be implemented using customizedcircuits, such as application specific integrated circuits (ASICs) orfield programmable gate arrays (FPGAs). In some embodiments, suchintegrated circuits execute instructions that are stored on the circuititself. In other embodiments, processing unit(s) 205 can executeinstructions stored in storage subsystem 210.

Storage subsystem 210 can include various memory units such as a systemmemory, a read-only memory (ROM), and a permanent storage device. TheROM can store static data and instructions that are needed by processingunit(s) 205 and other modules of electronic device 200. The permanentstorage device can be a read-and-write memory device. This permanentstorage device can be a non-volatile memory unit that storesinstructions and data even when computer system 200 is powered down.Some embodiments of the invention can use a mass-storage device (such asa magnetic or optical disk or flash memory) as a permanent storagedevice. Other embodiments can use a removable storage device (e.g., afloppy disk, a flash drive) as a permanent storage device. The systemmemory can be a read-and-write memory device or a volatileread-and-write memory, such as dynamic random access memory. The systemmemory can store some or all of the instructions and data that theprocessor needs at runtime.

Storage subsystem 210 can include any combination of computer readablestorage media including semiconductor memory chips of various types(DRAM, SRAM, SDRAM, flash memory, programmable read-only memory) and soon. Magnetic and/or optical disks can also be used. In some embodiments,storage subsystem 210 can include removable storage media that can bereadable and/or writeable; examples of such media include compact disc(CD), read-only digital versatile disc (e.g., DVD-ROM, dual-layerDVD-ROM), read-only and recordable Blue-Ray® disks, ultra densityoptical disks, flash memory cards (e.g., SD cards, mini-SD cards,micro-SD cards, etc.), magnetic “floppy” disks, and so on. The computerreadable storage media do not include carrier waves and transitoryelectronic signals passing wirelessly or over wired connections.

In some embodiments, storage subsystem 210 can store one or moresoftware programs to be executed by processing unit(s) 205, such as anapplication 235. “Software” refers generally to sequences ofinstructions that, when executed by processing unit(s) 205 causecomputer system 200 to perform various operations, thus defining one ormore specific machine implementations that execute and perform theoperations of the software programs. The instructions can be stored asfirmware residing in read-only memory and/or applications stored inmagnetic storage that can be read into memory for processing by aprocessor. Software can be implemented as a single program or acollection of separate programs or program modules that interact asdesired. Programs and/or data can be stored in non-volatile storage andcopied in whole or in part to volatile working memory during programexecution. From storage subsystem 210, processing unit(s) 205 canretrieve program instructions to execute and data to process in order toexecute various operations described herein.

A user interface can be provided by one or more user input devices 215,display device 220, and/or and one or more other user output devices(not shown). Input devices 215 can include any device via which a usercan provide signals to computing system 200; computing system 200 caninterpret the signals as indicative of particular user requests orinformation. In various embodiments, input devices 215 can include anyor all of a keyboard, touch pad, touch screen, mouse or other pointingdevice, scroll wheel, click wheel, dial, button, switch, keypad,microphone, and so on.

Display 220 can display images generated by electronic device 200 andcan include various image generation technologies, e.g., a cathode raytube (CRT), liquid crystal display (LCD), light-emitting diode (LED)including organic light-emitting diodes (OLED), projection system, orthe like, together with supporting electronics (e.g., digital-to-analogor analog-to-digital converters, signal processors, or the like). Someembodiments can include a device such as a touchscreen that function asboth input and output device. In some embodiments, other user outputdevices can be provided in addition to or instead of display 220.Examples include indicator lights, speakers, tactile “display” devices,printers, and so on.

In some embodiments, the user interface can provide a graphical userinterface, in which visible image elements in certain areas of display220 are defined as active elements or control elements that the userselects using user input devices 215. For example, the user canmanipulate a user input device to position an on-screen cursor orpointer over the control element, then click a button to indicate theselection. Alternatively, the user can touch the control element (e.g.,with a finger or stylus) on a touchscreen device. In some embodiments,the user can speak one or more words associated with the control element(the word can be, e.g., a label on the element or a function associatedwith the element). In some embodiments, user gestures on atouch-sensitive device can be recognized and interpreted as inputcommands; these gestures can be but need not be associated with anyparticular area in display 220. Other user interfaces can also beimplemented.

Network interface 225 can provide voice and/or data communicationcapability for electronic device 200. In some embodiments, networkinterface 225 can include radio frequency (RF) transceiver componentsfor accessing wireless voice and/or data networks (e.g., using cellulartelephone technology, advanced data network technology such as 3G, 4G orEDGE, WiFi (IEEE 802.11 family standards, or other mobile communicationtechnologies, or any combination thereof)), GPS receiver components,and/or other components. In some embodiments, network interface 225 canprovide wired network connectivity (e.g., Ethernet) in addition to orinstead of a wireless interface. Network interface 225 can beimplemented using a combination of hardware (e.g., antennas,modulators/demodulators, encoders/decoders, and other analog and/ordigital signal processing circuits) and software components. In someembodiments, network interface 225 can provide electronic device 200 aninterface to other computer systems, devices, and networks. For example,network interface 225 can enable electronic device 200 (e.g., atouchscreen device such as touchscreen device 102 of FIG. 1, a displaydevice such as display device 104 of FIG. 1) to connect to one or moredevices (e.g., a touchscreen device or a display device) via a wiredconnection (e.g., via a USB) or a wireless network (e.g., Bluetooth®).

Bus 230 can include various system, peripheral, and chipset buses thatcommunicatively connect the numerous internal devices of electronicdevice 200. For example, bus 230 can communicatively couple processingunit(s) 205 with storage subsystem 210. Bus 230 also connects to inputdevices 215 and display 220. Bus 230 also couples electronic device 200to a network through network interface 225. In this manner, electronicdevice 200 can be a part of a network of multiple computer systems(e.g., a local area network (LAN), a wide area network (WAN), anIntranet, or a network of networks, such as the Internet. Any or allcomponents of electronic device 200 can be used in conjunction with theinvention.

Some embodiments include electronic components, such as microprocessors,storage and memory that store computer program instructions in acomputer readable storage medium. Many of the features described in thisspecification can be implemented as processes that are specified as aset of program instructions encoded on a computer readable storagemedium. When these program instructions are executed by one or moreprocessing units, they cause the processing unit(s) to perform variousoperation indicated in the program instructions. Examples of programinstructions or computer code include machine code, such as is producedby a compiler, and files including higher-level code that are executedby a computer, an electronic component, or a microprocessor using aninterpreter.

Through suitable programming, processing unit(s) 205 can provide variousfunctionality for electronic device 200. For example, processing unit(s)205 can execute an application 235 that is configured to generatemultiple application user interfaces for presentation on multipledisplay devices. Examples of applications that can be stored andexecuted in storage subsystem 210 include productivity applications(e.g., e-mail applications, word processing applications, presentationcreation application, etc.), multi-media applications (e.g.,media-editing applications, media player applications, etc.), browserapplications, video game applications, or any other type of softwareapplication that can be operated via a user interface.

In some embodiments, application 235 can generate a first user interfaceconfigured to be presented on display 220. Application 235 can furthergenerate a second user interface configured to be presented on anotherdisplay of another electronic device that is in communication withcomputer system 200 (e.g., via wired or wireless communication link). Insome embodiments, application 235 can be configured to receive usercommands for interacting with at least one of the first or second userinterfaces (e.g., via user input devices 215 and/or user input devicesof the other electronic device) and process the user commands usingprocessor 205.

Application 235 can generate multiple user interfaces that include adifferent set of information items (e.g., a partial mirroring, a loweror higher level of abstraction) for display on the multiple displays. Insome embodiments, application 235 generates the user interfaces for eachdisplay in a way that would present content to the user in an intuitivemanner and that would provide better use of screen space across theavailable displays. For instance, an instant messaging application maygenerate a soft or virtual keyboard for display on the touch-enableddisplay of a smartphone. In addition, the application may generate auser interface that shows a text box with characters being entered bythe user (e.g., via the soft keyboard), along with the conversationexchange between the user and a participant, for display on a monitor.In some embodiments, the user interfaces generated by application 235can be interactive, such that user input received with respect to one ofthe user interfaces can affect the display of another user interface.Further, user input received with respect to one of the user interfacescan be used to control or interact with application 235 in someembodiments.

It will be appreciated that computer system 200 is illustrative and thatvariations and modifications are possible. Computer system 200 can haveother capabilities not specifically described here (e.g., mobile phone,global positioning system (GPS), power management, one or more cameras,various connection ports for connecting external devices or accessories,etc.). Further, while computer system 200 is described with reference toparticular blocks, it is to be understood that these blocks are definedfor convenience of description and are not intended to imply aparticular physical arrangement of component parts. Further, the blocksneed not correspond to physically distinct components. Blocks can beconfigured to perform various operations, e.g., by programming aprocessor or providing appropriate control circuitry, and various blocksmight or might not be reconfigurable depending on how the initialconfiguration is obtained. Embodiments of the present invention can berealized in a variety of apparatus including electronic devicesimplemented using any combination of circuitry and software.

FIG. 3 illustrates a simplified block diagram of a system 300 inaccordance with some embodiments of the present invention. System 300includes a touchscreen device 302 (e.g., similar to touchscreen device102 of FIG. 1) and display device 304 (e.g., similar to display device104 of FIG. 1). Specifically, touchscreen device 302 may be a tabletcomputer and display device 304 may be a computer monitor in thisexample.

Touchscreen device 302 can include a processor 306 (e.g., similar toprocessor 205 of FIG. 2), a storage device 308 (e.g., similar to storagesubsystem 210 of FIG. 2), a display 310 (e.g., similar to display 220 ofFIG. 2), touchscreen device 312 (e.g., similar to user input devices 215of FIG. 2), and a display device input/output interface 314 (e.g.,similar to network interface 225 of FIG. 2). Although not shown,touchscreen device 302 can include other components to provide variouscapabilities. For example, in some embodiments, touchscreen device 302can include transceiver components for wireless voice and/or datanetworks, a GPS receiver, and/or other components.

As described above, processor 306 can send and receive data or executevarious application programs stored in storage device 308. Processor 306can also manage communication with other electronic devices (e.g.,display device 304) via input/output (I/O) interface 314. As describedin further detail below, in one embodiment, processor 306 can execute aprogram application 316 such as an e-mail application for composingmessages to others, an instant messaging application, a media playerapplication for playing media assets and/or navigating a media assetdatabase, a video gaming application, a presentation application, a mapapplication for providing directions, a web browser application forbrowsing the Internet, a video conferencing application, a photographmanipulation software application, etc. Further, in some embodiments,storage device 208 can store media assets such as audio, video, stillimages, or the like, and associated metadata (e.g., asset name, artist,title, genre, playlists, etc.). Storage device 208 can also store auser's address book information including a user's contacts (names,addresses, phone numbers, etc.); a user's calendar information such asscheduled appointments and events; notes; and/or other personalinformation.

In this example, touchscreen device 302 can be a tablet device thatincorporates a touchscreen 312 configured to perform the functions of akeyboard/mouse. Touchscreen device 302 can include additional devicesthat allow a user to interact with touchscreen device 302, as describedwith respect to user input devices 215 of FIG. 2. In some embodiments,the user can operate a touchscreen 312 to interact with contentdisplayed on touchscreen device 302 and display device 304. Display 310can be an integral part of touchscreen device 302, such as an embeddedLCD or OLED panel. Like display 310, touchscreen 312 can be atouch-based interface that is integrated into a display screen or othersurface of touchscreen device 302. Similar to network interface 225 ofFIG. 2, device I/O interface 314 can serve as an interface forcommunicating data between touchscreen device 302 and other devices ornetworks. In various embodiments, device I/O interface 314 can be awired (e.g., twisted pair Ethernet, USB, etc.) or wireless (e.g., WiFi,cellular, etc.) interface.

Display device 304 can include a processor 320, a display 322, and atouchscreen device I/O interface 318. Display device 304 isrepresentative of a broad range of electronic devices to whichtouchscreen device 302 can be connected, and it is understood that suchdevices can vary widely in capability, complexity and form factor.Specifically, display device 304 in this example can be representativeof a computer monitor coupled to touchscreen device 302 (e.g., a tabletcomputer). Various electronic devices may include components not shownin FIG. 3, including but not limited to storage devices (disk, memory,etc.), ports for connecting to external speakers and/or display devices,user input devices, additional output devices, and so on.

Processor 320 can be implemented as one or more integrated circuits,such as a microprocessor or microcontroller. In various embodiments,processor 320 can execute program code to perform various functionsassociated with display device 304. For example, in some embodimentswhere display device 304 incorporates a sound system (e.g., speakersystem), program code executed by processor 320 can include programs fordigital audio decoding, analog or digital audio processing, and thelike. Processor 320 can also manage communication with touchscreendevices (e.g., touchscreen device 302) via I/O interface 318.

Display 322 can be any type of device capable of receiving information(e.g., display signals) from display device 304 and outputting thereceived information on a screen or other output interface to a user. Inone set of embodiments, display device 304 can be a computer monitor anddisplay 322 is an integral part of display device 304, such as anembedded LCD or OLED panel. In some embodiments, display 322 can includean audio output device for presenting audio (in addition to images andvideo) to a user. Alternatively, display device 304 can be a computerdesktop and display 322 can be external to display device 304. Forinstance, display 322 can be a computer monitor, a television, or someother type of standalone display that is in wired or wirelesscommunication with display device 304.

Touchscreen device I/O interface 318 can include a number of signalpaths configured to carry various signals between display device 304 andtouchscreen device 302. In one set of embodiments, touchscreen deviceI/O interface 318 includes a connector or a wireless interface (e.g.,Bluetooth or the like).

It should be appreciated that system 300 is illustrative and notintended to limit embodiments of the present invention. For example,touchscreen device 302 and display device 304 may each have othercapabilities or include other components that are not specificallydescribed. One of ordinary skill in the art will recognize manyvariations, modifications, and alternatives.

FIG. 4 illustrates another simplified block diagram of a system 400 inaccordance with some embodiments of the present invention. As shown,system 400 includes various components/features that are substantiallysimilar to system 300 of FIG. 3, but application 416 is stored andexecuted on display device 402 instead of on touchscreen device 404.Electronic device 402 may be a desktop computer in this example, whileportable electronic device 404 may be a tablet computer. Portableelectronic device 404 still incorporates a touchscreen 412 that isconfigured to receive user input. Electronic device 402 employs aprocessor 406 that can carry out one of more functions attributable toelectronic device 402, such as executing application 416. Processor 406can also manage communication with touchscreen device 404.

In some embodiments, an application executing on one device can drivethe displays of both a touchscreen device and a display device whilereceiving user input from the touchscreen device. FIG. 5 illustrates anexample of a process 500 performed by a touchscreen device (e.g.,touchscreen device 102) for interacting with a display device (e.g.,display device 104) having a non-touch display according to someembodiments of the present invention.

Process 500 begins when the touchscreen device establishes a connection(at block 502) with the display device. As mentioned, the touchscreendevice can be an electronic device such as a tablet computer with atouch-enabled display. The display device can be a desktop computer or amonitor with a non-touch display. In some embodiments, establishingcommunication between the devices 102 and 104 may include identifyingand authenticating display device 104 (and vice versa) using a set ofcommands. Communication between the devices may be established through awired (e.g., via a cable) or wireless connection (e.g., via Bluetooth,via near field communication (NFC)).

Process 500 can present (at block 504) a first set of information itemson the touch-enabled display of the touchscreen device. In someembodiments, touchscreen device 102 may launch an application when auser indicates the desire to use tools and functionality provided by theapplication. The application can present a set of information items onthe touch-enabled display where the first set of information items isnavigable and selectable by a user of the touchscreen device. Forexample, upon launching a media playing computer program, thetouch-enabled display may present a user interface including a list ofselectable user interface items representing a set of songs. In anotherexample, a mapping application can present a navigable map where theuser can use controls on the display of the portable electronic deviceto view different aspects or portions of the map.

Process 500 can receive (at block 506) a selection of one of the firstset of information items from the user via the touch-enabled display ofthe touchscreen device. The first set of information items may include aset of selectable user interface items representing a set of songs for amedia player application, a set of e-mails for an e-mail clientapplication, a set of controls for a video gaming application, acombination of a set of editing tools and a series of presentationslides, a set of photos, a virtual keyboard, etc. The user may selectone or more selectable user interface items via the touchscreen ortouch-enabled display of the touchscreen device.

Process 500 can enable (at block 508) a second set of information itemsassociated with the selected information item to be presented on thedisplay of the second electronic device while the first set ofinformation items is presented on the touch-enabled display of the firstelectronic device. In some embodiments, the second set of informationitems represents an additional level of information (e.g., a higherlevel of information, a lower level of information) related to theselected information item. For example, upon receiving user selection ofa particular e-mail from a list of e-mails being displayed on thetouch-enabled display of touchscreen device 102, the application cancause display device 104 to display a lower level of informationincluding content of the particular e-mail.

In some embodiments, the first set of information items displayed ontouchscreen device 102 can include a set of selectable control elements.The user can select at least one of the set of control elements toindicate a change to content of the application. For instance, in ane-mail client application, the user can select a “compose” controlelement on touchscreen device 102 and view a composition window on otherdisplay. Then, when the user types a character, it would appear in thecomposition window. The user can edit content of an e-mail by performingcharacter input via a virtual keyboard displayed on the user interfaceof touchscreen device 102. In response to receiving the user input, theapplication can cause the user interface on display device 104 todisplay the characters input by the user. In this instance, the secondset of information items displayed on display device 104 can represent ahigher level of information related to the selected information item(i.e., selectable keys on the virtual keyboard representing characters)by displaying the entirety of the e-mail and the content being inserted.

In some embodiments, the first and second set of information items maybe identical or similar. In response to receiving the user selection ofan information item, the application may generate another user interfacethat is a mirror image of the user interface displayed on thetouch-enabled display of the first electronic device. In someembodiments, the application may generate a partial mirror-image fordisplay on the display of the second electronic device. For instance, apresentation application may display a set of tools, a set ofpresentation slides, and an image of the selected slide on thetouch-enabled display while causing the display of the second electronicdevice to display only the selected slide. In response to receiving aselection of the slide, the application can cause display device 104 topresent a lower level of information including a zoomed-in view of theslide. In some instances, the application may further cause displaydevice 104 to display additional details about the selected slide.

While some embodiments may store and execute the application ontouchscreen device 102, the application in some embodiments may bestored and executed on display device 104. Regardless of which devicestores and executes the application, the application can generatemultiple interactive user interfaces that can be displayed across themultiple devices. FIG. 6 illustrates an example of a process 600performed by a first electronic device (e.g., electronic device 104) forpresenting information across the first electronic device and a secondelectronic device (e.g., portable electronic device 102) in accordancewith some embodiments of the invention.

As described, the application may be executed on a display device (e.g.,display device 104 of FIG. 1) while receiving user input from atouchscreen device (e.g., touchscreen device 102 of FIG. 1). Process 600begins when the touchscreen device and the display device establishcommunication (at block 602). As described, the devices can establishcommunication via a wired or wireless connection. Process 600 can send(at block 604) a first set of information items to a touchscreen devicewith a touch-enabled display. An application being launched on thedisplay device can generate a user interface for display on the displaydevice and another user interface for display on the touch-enableddisplay of the touchscreen device. Upon generating the other userinterface including a set of information items, the application can sendthe set of information items to the touchscreen device for display.

Process 600 can receive (at block 606) user interaction with the set ofinformation items from the touchscreen device. The user may perform userinput via the touch-enabled display of touchscreen device 102. In someembodiments, the user input may indicate a selection of a particularitem in the set of information items. For example, the application maycause touchscreen device 102 to display a list of restaurants, a set ofmedia player control elements, or a number of keys of a virtualkeyboard. The user may then select a particular restaurant, a particularcontrol element, or a particular key from the list of selectable itemsdisplayed.

Process 600 can display (at block 608) another set of information itemson a display of the touchscreen device in response to receiving the userinteraction. In some embodiments, the other set of information itemsrepresents an additional level of information associated with at leastone of the set of information items. For example, in response toreceiving user selection of a particular restaurant, a restaurantrecommendation application may generate a user interface that includesanother set of information items (e.g., detailed information about therestaurant, directions to the restaurant, hours, pictures, etc.) andcause the display device to display the user interface including theother set of information items. In another example, in response toreceiving user selection of a particular song, a media playerapplication can generate a user interface that includes another set ofinformation items, including album art, frequency characteristics of theparticular song, etc., for display on the display device.

It should be appreciated that processes 500 and 600 are illustrative andthat variations and modifications are possible. For example, althoughprocess 500 indicates that application 235 (executing on eithertouchscreen device 102 or display device 104) is configured to performthe tasks of generating user multiple user interfaces, processing userinput commands, and generating user interfaces in response to thecommands, in alternative embodiments, some portion of these tasks can beperformed by either the other device (e.g., touchscreen device 102,display device 104) or by other remote computing devices (e.g., remoteservers).

Further, some or all of the processes 500 and 600 (or any otherprocesses described herein, or variations and/or combinations thereof)may be performed under the control of one or more computer systemsconfigured with executable instructions and may be implemented as code(e.g., executable instructions, one or more computer programs, or one ormore applications) executing collectively on one or more processors, byhardware, or combinations thereof. The code may be stored on acomputer-readable storage medium, for example, in the form of a computerprogram to be executed by processing unit(s), such as a browserapplication. The computer-readable storage medium may be non-transitory.

FIGS. 7-17B illustrate examples of user interfaces generated for displayacross a touch-enabled display of a touchscreen device (e.g.,touchscreen device 102 of FIG. 1) and non-touch display of a displaydevice (e.g., display device 104 of FIG. 1). As described, anapplication (e.g., application 235 of FIG. 2) can be stored and executedon at least one of a touchscreen device (e.g., tablet computer,smartphone) and a display device (e.g., desktop, monitor, laptop) andcan generate and modify multiple user interfaces displayed across thedevices based on user input. The user interfaces may be presented acrossthe devices in a coordinated fashion. Content displayed in the each ofthe user interfaces may include different aspects the application,thereby facilitating interaction with the application. In someembodiments, the user interfaces may display different levels ofinformation in the application.

FIG. 7 illustrates an example 700 of how different levels of informationcan be displayed on a display device 704 and a touchscreen device 702 inaccordance with some embodiments of the invention. In certainembodiments, application 235 of FIG. 2 can be a digital photographmanipulation application that is configured to generate a first userinterface for presentation on touch-enabled display 706 and a seconduser interface for presentation on computer monitor 708. As shown inthis example, the first user interface for presentation on touch-enableddisplay 706 includes a set of thumbnails, each representing a photo. Thesecond user interface displays additional detailed information about aparticular photo that has been selected by the user.

In this instance, the thumbnail representing a photo of Clarissa Ramsayhas been selected, and the application has generated the second userinterface for display on computer monitor 708. The second user interfaceincludes an image represented by the thumbnail and information about thephoto, the date/time when the photo was taken, the album name andnumber, the location, event, and additional information that applicationhad captured when the photo was taken or that the user had recorded. Theuser may view the different thumbnails in a particular album, such asthe “Fall 2012 Classmates” album and select to view the details of anyof the thumbnails upon performing a selection through touch-enabled userinterface of touchscreen device 702. In this instance, the user mayselect the thumbnails at random order for viewing on the larger computermonitor display 708, and need not scroll through the thumbnails in aparticular order.

In some embodiments, in response to receiving a selection of theparticular thumbnail, the application can generate the second userinterface that includes an enlarged display of the image correspondingto the selected thumbnail. The second user interface may be displayed oncomputer monitor 704 without additional information. In this instance,the user of touchscreen device 702 can view the set of thumbnails (orlist of files including a filename and/or an icon with an image) ontouchscreen device 702 while spontaneously selecting images for viewingon computer monitor 704 in a randomized order. This allows the user topresent a slideshow customized to the current audience withoutpre-defining the order of the slideshow. In some embodiments, uponselecting the particular thumbnail, computer monitor 704 may display thedetailed information related to the particular thumbnail and not thephoto represented by the thumbnail. Different embodiments may generatedifferent user interfaces for display across the devices in a mannerthat is intuitive to the user.

FIG. 8 illustrates another example 800 of how different levels ofinformation can be displayed on a display device 804 and a touchscreendevice 802 in accordance with some embodiments of the invention. In thisexample, at least one of display device 804 and touchscreen device 802is running a video game application. The video game applicationgenerates a user interface including video game scenes for display ondisplay device 804. The video game application further generates anotheruser interface including a set of control elements for display ontouchscreen device 802.

Through a touch-enabled display 806 of touchscreen device 802, the usercan select various control elements to control the movement and actionsof the player. In this example, when the user selects a particularcontrol element (e.g., a firing control), the player in the video gameapplication performs a firing action, which can be seen in the userinterface generated for display on display device 804. In response toreceiving a user input via touch-enabled display of touchscreen device802, the video game application generates another user interface fordisplay on a non-touch display 808 of display device 804 that isdetermined based on the user interaction with the user interfacedisplayed on touchscreen device 802.

While the gaming application in this example causes controls to bedisplayed on the touchscreen device and visual feedback to be displayedon the display device, a DJ application or an application that allowsusers to create music can also cause the controls to be displayed on thetouchscreen device while album art or a music score is displayed on thedisplay device. For instance, the DJ application can display a set ofcontrols including a selectable surface of a drum, piano keyboard, etc.to be displayed on the touchscreen device. The user may then createmusic or cause album art or visualizer to be displayed on the displaydevice by performing a series of user interaction with the set ofcontrols.

FIG. 9 illustrates another example 900 of how different levels ofinformation can be displayed on a display device 904 and a touchscreendevice 902 in accordance with some embodiments of the invention. In thisexample, a map application is being executed on at least one of displaydevice 904 and touchscreen device 902. The map application can generateviews of different perspectives and for different levels of detail of aparticular map area on devices 902 and 904 in response to a usercommand. As shown in this example, touchscreen device 902 shows a bird'seye view of a map area while display device 904 shows a street levelperspective view of a particular location within the map area.

In some embodiments, touchscreen device 902 may further display a set ofselectable controls that the user can select to indicate the directionof the street view to display on display device 904. Additional levelsof mapping information can be provided by the user interface displayedon display device 904 as well, including a list of stores, restaurants,and gas stations around a particular area indicated by the user (e.g.,via a selection of a particular location on the map on the userinterface displayed on touchscreen device 902), or other types ofinformation that can be provided about the particular location.

FIG. 10 illustrates another example 1000 of how different levels ofinformation can be displayed on a display device 1004 and a touchscreendevice 1002 in accordance with some embodiments of the invention. Inthis example, a video conference application is running on displaydevice 1004. The application generates a first user interface includinga video image of the local user for display on touchscreen device 1002.The application further generates a second user interface including avideo image of another participant (or multiple other participants) withwhom the user is having the conference for display on display device1004. By presenting an arrangement of the video images that would beintuitive to a user (e.g., where the user's local video is shown ontouchscreen device 1002 and the remote participant's feed is viewablethrough electronic device 1004), the application can provide the user abetter user experience in using the application.

In some embodiments, a number of control elements 1006 (e.g., menus,slider bars, text fields, and buttons representing volume adjustments,lighting adjustments, focus adjustments, and/or zoom adjustmentsfunctionality) can be displayed on touchscreen device 1002 such that theuser may make adjustments when desired by performing a user inputthrough the touch-enabled display of touchscreen device 1002. As such,the remote video need not be obstructed by control elements. Controls1006 may also fade out in some embodiments when the user has notinteracted with the touch-enabled display beyond a threshold period oftime (e.g., after 5 seconds), thereby increasing the screen real estatefor the user's video image.

FIG. 11 illustrates another example 1100 of how different levels ofinformation can be displayed on a display device 1104 and a touchscreendevice 1102 in accordance with some embodiments of the invention. Inthis example, a presentation application is running on touchscreendevice 1102. The presentation application generates a first userinterface 1106 including a series of presentation slides 1108 that arein a file, a set of controls 1110, and an image of a slide 1112 that iscurrently selected by the user for display on touchscreen device 1102.The presentation application further generates a second user interface1114 including a mirror image of the selected slide for display ondisplay device 1104.

Through the touch-enabled display of touchscreen device 1102, the usercan perform a set of actions while giving the presentation, such asselecting text or items within the slide currently being displayed,highlighting or circling portions of the slide (currently shown), orwriting on the slide. In response to receiving the user input, thepresentation application can generate a partial mirror image of the userinterface being displayed on touchscreen device 1102 for display ondisplay device 1104. The partial mirror image 1114 can include the slidecurrently selected for display on touchscreen device 1102 and the userinput (i.e., a portion being circled) being performed on the slide.

In some embodiments, the presentation application can allow users tocreate and edit presentations. In response to user indication that theuser would like to edit a presentation slide, the application can causethe big screen or display device 1104 to display the full slide andcause touchscreen device 1102 to display a close-up of the portion theuser is currently editing. A set of editing tools and a virtual keyboardmay also be displayed on touchscreen device 1102 in addition to theclose-up of the portion that the user is currently editing.

FIG. 12 illustrates another example 1200 of how different levels ofinformation can be displayed on a display device 1204 and a touchscreendevice 1202 in accordance with some embodiments of the invention. Inthis example, a media player application is running on touchscreendevice 1202. The media player application generates a first userinterface 1206 including a set of music albums 1208 that are viewableand selectable by the user on touchscreen device 1202. The media playerfurther generates a second user interface 1210 including detailedinformation about one of the music albums selected by the user (as shownby highlighted box 1205) for display on display device 1204. As shown inthis example, a set of information items 1212 (i.e., lower level ofinformation) are displayed on display device 1204, including album art,information about the artist, a list of songs included in the selectedalbum, and characteristics related to each song, including a length ofthe song.

Through the touch-enabled display of touchscreen device 1202, the usercan randomly select from album to album from the displayed set ondisplay device 1204 to view the details under each album. In response toreceiving the user input, the media player application can generatesecond user interface 1210 including detailed information or a lowerlevel of information of the selected item. As such, the user maymaintain a view of all the available albums while being able to view thedetails of a selected album at the same time.

FIG. 13 illustrates another example 1300 of how different levels ofinformation can be displayed on a display device 1304 and a touchscreendevice 1302 in accordance with some embodiments of the invention. Inthis example, a browser application is running on touchscreen device1302. As described above, the browser application can be running oneither touchscreen device 1302 or display device 1304. The browserapplication can generate a first user interface 1306 on touchscreendevice 1302 that includes a list of the user's bookmarked websites orfavorite websites 1308. The browser application can also generate asecond user interface 1310 including the contents of a selected website(i.e., lower level details of an information item) for display ondisplay device 1304. In some embodiments, this may include an article, amultimedia item such as a video file, or any other objects that may beincluded in a webpage. In this example, the content of a selectedwebsite (i.e., a video broadcast of a news station) is a lower level ofinformation related to the selected website.

Through the touch-enabled display of touchscreen device 1302, the usercan launch a browser application and identify sites that the user wouldlike to visit. The browser application may display a set of the user'sfavorite website or bookmarked websites on device 1302 upon launchingthe application and allow the user to select a website to be displayedon display device 1304. In response to receiving the user input, thebrowser application can cause display device 1304 to load and displaythe website corresponding to the selected link 1312. A set of controlbuttons 1314 may also be displayed on touchscreen device 1302 to enablethe user to request to return to the previous website or to the nextwebsite. Further, in response to receiving a user selection of anotherwebsite in the bookmarked websites, touchscreen device 1302 may causedisplay device 1304 to load and display contents of the other website.As such, the different images generated and displayed on the devices1302 and 1304 are interactive and responsive to user input totouchscreen device 1302.

FIG. 14 illustrates another example 1400 of how different levels ofinformation can be displayed on a display device 1404 and a touchscreendevice 1402 in accordance with some embodiments of the invention. Inthis example, a page layout application (also referred to as wordprocessing application) is running on touchscreen device 1402. Asdescribed above, the page layout application can be running on eithertouchscreen device 1402 or display device 1404. The page layoutapplication can generate a first user interface 1406 including aneditable region 1408, a set of editing tools 1410, and a virtualkeyboard 1412 for display on touchscreen device 1402. The page layoutapplication can also generate a second user interface 1414 including aset of pages included within a document/file for display on displaydevice 1404. Editable region 1408 can include a subset of the pagesincluded within the document. In this example, editable region 1408includes a single page from the set of pages being displayed on displaydevice 1404.

Through the touch-enabled display of touchscreen device 1402, the usercan select a portion of the document that the user would like to focuson editing. The page layout application can provide a set of editingtools 1410 that the user may select and use to edit the portion of thedocument that is being displayed in editable region 1408. The user mayalso perform character input using virtual keyboard 1412 displayed ontouchscreen device 1402. While the user is editing a portion of thedocument, the user may view the entire document or multiple pages of thedocument on display device 1404. By displaying a higher level ofinformation (or by zooming out such that multiple pages of the documentare viewable by the user, the user can see the big picture as to what isincluded in the document and how the document look as a whole, while theuser edits a portion of the document. In some embodiments, the user'sedits in touchscreen device 1402 may be reflected on display device 1404in real-time or in response to user edits.

While in this example the document includes multiple pages, in someembodiments only a single page is included in the document (e.g., aposter page, a postcard). In some embodiments, the user may be editing aportion of a page (e.g., half a page, a quarter of a page) ontouchscreen device 1402 while the entirety of the page is displayed ondisplay device 1404. As such, the user may perform edits to portions ofthe page on touchscreen device while simultaneously viewing how theedits are reflected on the page when the entire page is in view ondisplay device 1404.

FIG. 15 illustrates another example 1500 of how different levels ofinformation can be displayed on a display device 1504 and a touchscreendevice 1502 in accordance with some embodiments of the invention. Inthis example, a messaging application (e.g., an instant messaging app)is running on touchscreen device 1502. As described above, the messagingapplication can be running on either touchscreen device 1502 or displaydevice 1504. The messaging application can generate a first userinterface 1506 including a virtual keyboard 1508 and a text entry box1510 where the user may enter a message to send to a recipient. Themessaging application can also generate a second user interface 1512including a communication exchange 1514 between the user and therecipient. In this example, the communication exchange between the userand the recipient is a higher level of information relative to theuser's entered message.

Through the touch-enabled display of touchscreen device 1502, the usercan compose messages using virtual keyboard 1508 and view the enteredtext simultaneously through the same display on touchscreen device 1502.In response to receiving the user input to send a message to a recipient(or receiving a message from a recipient), the messaging application maycause display device 1504 to update the communication exchange. As thecommunication exchange is constantly updated and displayed on displaydevice 1504, the user may have a more comprehensive view of thecommunication exchange on a larger display of display device 1504. Inthis example, a higher level of information (i.e., content of theconversation exchange) is displayed on the larger display of the displaydevice while a lower level of information (i.e., the user's message) andcontrol elements including virtual keyboard 1508 are displayed on thetouchscreen device. Different combination may also be available indifferent embodiments.

FIGS. 16A-16B illustrate another example 1600 of how different levels ofinformation can be displayed on a display device 1604 and a touchscreendevice 1602 in accordance with some embodiments of the invention. Inresponse to user interaction with touchscreen device 1602, differentlevels of information can be generated for each device and displayed toa user in an intuitive manner. In this example, an e-mail applicationcan be running on either touchscreen device 1602 or display device 1604.The e-mail application can generate a first user interface 1606including a list of e-mail messages 1608 in the user's inbox includingsender information, a subject of each e-mail, and a portion of thecontent of each e-mail. The e-mail application may further generate asecond user interface 1614 that includes a larger version (or zoomed-inversion) of the contents of a selected e-mail 1612. In this example, thecontents of the e-mail message displayed on display device 1604represents a “lower level of information” as first user interface 1606includes the list of the e-mail messages 1608 and various controlelements 1616 in addition to the contents of selected e-mail 1612.

Through the touch-enabled display of touchscreen device 1602, the usercan navigate through the list of e-mails and select a preview of ane-mail in e-mail viewing pane 1608. In response to receiving a userselection of a particular e-mail 1612 for viewing, the e-mailapplication may cause display device 1604 to display the contents ofemail 1612. As such, the user may view the contents of different e-mailsby selecting the e-mails in the list (in a non-sequential order or inorder). While FIG. 16A illustrates the one combination of the differentlevels of information that can be displayed on device 1602 and 1604 inresponse to user interaction with touchscreen device 1602, FIG. 16Billustrates that in response to different user input, the e-mailapplication may generate another level of information for display ondisplay device 1604. That is, the user interface displayed on eachdevice can be interactive and responsive to the user input.

In FIG. 16B, the user begins typing a response to the selected e-mail(after hitting reply key 1610 shown in FIG. 16A). As shown in thisexample, the reply message that is being typed by the user (in text box1614) is simultaneously displayed on display device 1604 such that theuser may view the totality of the chain message (including the responseand the e-mail that is being responded to) while the user types themessage using a virtual keyboard 1616 and editing/formatting tools 1618(e.g., font, size, color, etc.) displayed on touchscreen device 1602. Inthis example, a higher level of information (i.e., content of the chaine-mail) is displayed on the larger display of the display device while alower level of information (i.e., the user's response) and variouscontrol elements including a virtual keyboard are displayed on thetouchscreen device.

While in FIG. 16A, the e-mail application causes touchscreen device 1602to display a list of e-mails and display device 1604 to display contentsof a selected e-mail and in FIG. 16B, the e-mail application causestouchscreen device 1602 to display a response message and variousselectable elements usable to create the response message and displaydevice 1604 to display a comprehensive overview of the e-mail chain, insome embodiments, the e-mail application can cause one of touchscreendevice 1602 and display device 1604 to display an e-mail message whilethe other (e.g., display device 1604) displays the content of one ormore attachments to the e-mail. In the instance where touchscreen device1602 displays the e-mail message while display device 1604 displays anattachment to the e-mail, the user may type a reply message displayed ontouchscreen device 1602 while viewing one or more of the attachments ondisplay device 1604.

FIGS. 17A-17B illustrate another example 1700 of how different levels ofinformation can be displayed on a display device 1704 and a touchscreendevice 1702 in accordance with some embodiments of the invention. Inthis example, a movie player application (or video editing application)can be running on either touchscreen device 1702 or display device 1704.The movie player application can generate a first user interface 1706including a set of movie thumbnails 1708 representing the user's moviecollection. The movie player application may further generate a seconduser interface 1710 that plays the contents of a selected movie 1712. Inthis example, the content of the video or movie displayed on displaydevice 1704 represents a “lower level of information.”

Through the touch-enabled display of touchscreen device 1702, the usercan navigate through the videos/movie thumbnails displayed ontouchscreen device 1702 and select a preview of a movie in displaydevice 1704. In response to receiving a user selection of a particularmovie 1712 to preview or to play, the movie player application may causedisplay device 1704 to display the contents of movie 1712. As such, theuser may view the contents of different movies as the user selectsdifferent videos in the list (in a non-sequential order or in order).

FIG. 17B illustrates that after the movie begins (e.g., after a certainamount of time or after the user has indicated that the user would liketo watch the selected movie), the movie player application may generateanother user interface 1714 for display on touchscreen device 1702. Inthis example, touchscreen device 1702 also displays the movie beingdisplayed on display device 1704. However, touchscreen device 1702 alsodisplays a set of controls 1716 (e.g., fast forward, pause, rewind,etc.) such that the user may control the movie being played on devices1702 and 1704. Additional controls such as a selectable play button, astop button, next/previous track buttons, an adjustable knob that theuser can select and scroll to change the scene currently displayed onthe display device, etc. may also be displayed in some embodiments. Insome embodiments, the set of controls 1716 may not be readily displayedand may appear in response to a user input (e.g., a tap on touchscreendevice 1702). In response to different user input (e.g., controlsignals), the movie player application may modify the informationdisplayed on display device 1704 (and touchscreen device 1702). That is,the user interface displayed on each device can be interactive andresponsive to the user input.

Further, the user may configure touchscreen device 1702 to displayclosed captioning or subtitles such that the user may have supplementalinformation to the movie accessible if the user so desires. This permitsviewers who prefer not to see the captions to not have to be forced towatch the movie with captions displayed on the display device. As such,the application may generate a number of user interfaces that would beintuitive for the user's use of multiple display screens. Theapplication can display the generated user interfaces in a displayarrangement (e.g., which user interface for which device) in a way thatmaximizes use of the multiple screens (a large screen and a smallertouchscreen).

It should be appreciated that FIGS. 7-17B are illustrative and notintended to limit embodiments of the present invention. For example,although an application is shown as generating two different userinterfaces in each figure, any number of such user interfaces can begenerated. In another example, although a touch device and a non-touchdevice are used in the examples, any electronic device can be usedregardless of whether the device is a touch device or a non-touchdevice. One of ordinary skill in the art will recognize othervariations, modification, and alternatives. Further, although particularexamples of applications and user interfaces generated by suchapplications are described above, other applications may be used togenerate user interfaces that can be displayed across multiple displaysin an intuitive manner.

In some embodiments, the device running the application can causeanother device (e.g., a computer desktop, a tablet computer) to displaya set of information items by providing a structured data file (e.g., aneXtensible Markup Language (XML) file) to the other device, and theother device (i.e., the device not running the application) can run anapplication-specific interpreter to render an image including the set ofinformation items. In some embodiments, the device running theapplication can cause the other device to display the set of informationitems by providing a pixel stream or pixel data (e.g., in digital oranalog form) to the other device, which then renders an image. Forexample, when the other device is a monitor, a controller of the monitormay read the pixel data and display an image based on the pixel data.

Embodiments of the present invention can be realized using anycombination of dedicated components and/or programmable processorsand/or other programmable devices. The various processes describedherein can be implemented on the same processor or different processorsin any combination. Where components are described as being configuredto perform certain operations, such configuration can be accomplished,e.g., by designing electronic circuits to perform the operation, byprogramming programmable electronic circuits (such as microprocessors)to perform the operation, or any combination thereof. Further, while theembodiments described above may make reference to specific hardware andsoftware components, those skilled in the art will appreciate thatdifferent combinations of hardware and/or software components may alsobe used and that particular operations described as being implemented inhardware might also be implemented in software or vice versa.

Computer programs incorporating various features of the presentinvention may be encoded and stored on various computer readable storagemedia; suitable media include magnetic disk or tape, optical storagemedia such as compact disk (CD) or DVD (digital versatile disk), flashmemory, and other non-transitory media. Computer readable media encodedwith the program code may be packaged with a compatible electronicdevice, or the program code may be provided separately from electronicdevices (e.g., via Internet download or as a separately packagedcomputer-readable storage medium).

Thus, although the invention has been described with respect to specificembodiments, it will be appreciated that the invention is intended tocover all modifications and equivalents within the scope of thefollowing claims.

What is claimed is:
 1. A method of displaying a maps application acrossmultiple displays, the method including: at a first device that is incommunication with a second device; generating for the maps applicationa first user interface and information for generating a second userinterface; displaying, at a first display of the first device, the firstuser interface associated with a location on a map; and transmitting theinformation for generating a second user interface to the second deviceso that a second display of the second device displays a second userinterface that includes the location on a map in the maps application,where the second user interface is different from the first userinterface.
 2. The method of claim 1, wherein the first display has afirst size smaller than a size of the second display.
 3. The method ofclaim 1, wherein the first device is a mobile phone and second device isnot a mobile phone.
 4. The method of claim 1, including: receiving aninput at the first user interface; and in response to receiving theinput at the first user interface, sending an update to the seconddevice to update the second user interface to present additionalinformation associated with the maps application.
 5. The method of claim4, wherein the additional information includes points of interestdisplayed within the maps application.
 6. The method of claim 5,including: receiving an input at one of the points of interest, and inresponse displaying additional information associated with the point ofinterest.
 7. The method of claim 1, wherein the first display is a touchsensitive display.
 8. The method of claim 1, wherein the second displayis not touch sensitive.
 9. The method of claim 1, wherein the first userinterface associated with the location on the map in the mapsapplication is a bird's eye view of the map.
 10. The method of claim 1,wherein the second user interface associated with the location on themap in the maps application is a bird's eye view of the map.
 11. Themethod of claim 1, wherein the first device is in wireless communicationwith the second device.
 12. A non-transitory computer readable storagemedium storing one or more programs, the one or more programs comprisinginstructions that, when executed by a first device with a first displaythat is in communication with a second device, cause the first deviceto: generate for the maps application a first user interface andinformation for generating a second user interface; display, at a firstdisplay of the first device, the first user interface associated with alocation on a map; and transmit the information for generating a seconduser interface to the second device so that a second display of thesecond device displays a second user interface that includes thelocation on a map in the maps application, where the second userinterface is different from the first user interface.
 13. Thenon-transitory computer readable storage medium of claim 12, wherein thefirst display has a first size smaller than a size of the seconddisplay.
 14. The non-transitory computer readable storage medium ofclaim 12, wherein the first device is a mobile phone and second deviceis not a mobile phone.
 15. The non-transitory computer readable storagemedium of claim 12, wherein the one or more programs includeinstructions that, when executed by the first electronic device, causethe first electronic device to: receive an input at the first userinterface; and in response to receiving the input at the first userinterface, send an update to the second device to update the second userinterface to present additional information associated with the mapsapplication.
 16. The non-transitory computer readable storage medium ofclaim 15, wherein the additional information includes points of interestdisplayed within the maps application.
 17. A first device that is incommunication with a second device, comprising: a first display; one ormore processors; memory; and one or more programs, wherein the one ormore programs are stored in the memory and configured to be executed bythe one or more processors, the one or more programs includinginstructions for: generating for the maps application a first userinterface and information for generating a second user interface;displaying, at the first display of the first device, the first userinterface associated with a location on a map; and transmitting theinformation for generating a second user interface to the second deviceso that a second display of the second device displays a second userinterface that includes the location on a map in the maps application,where the second user interface is different from the first userinterface.
 18. The first device of claim 17, wherein the first displayhas a first size smaller than a size of the second display.
 19. Thefirst device of claim 17, wherein the first device is a mobile phone andsecond device is not a mobile phone.
 20. The first device of claim 17,wherein the one or more programs include instructions for: receiving aninput at the first user interface; and in response to receiving theinput at the first user interface, sending an update to the seconddevice to update the second user interface to present additionalinformation associated with the maps application.